@*
* Copyright 2016 LinkedIn Corp.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
* use this file except in compliance with the License. You may obtain a copy of
* the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under
* the License.
*@

<p>
    This analysis shows the task memory utilization. <br>
    We check the ratio between your task's consumed memory AND the requested container memory. <br>
    The consumed memory is the average of each task's [max consumed physical memory snapshot]. <br>
    The requested container memory is the "mapreduce.map/reduce.memory.mb" config for this job, which is the max physical memory the job can request. <br>
    If this heuristic is above MODERATE, it means your used memory is relatively low compared with the requested max memory.
</p>

<h5>Example</h5>
<p>
<div class="list-group">
    <a class="list-group-item list-group-item-danger" href="#">
        <h4 class="list-group-item-heading">Mapper Memory</h4>
        <table class="list-group-item-text table table-condensed left-table">
            <thead><tr><th colspan="2">Severity: Critical</th></tr></thead>
            <tbody>
            <tr>
                <td>Number of tasks</td>
                <td>1000</td>
            </tr>
            <tr>
                <td>Avg Physical Memory (MB)</td>
                <td>512</td>
            </tr>
            <tr>
                <td>Requested Containter Memory</td>
                <td>4G</td>
            </tr>
            </tbody>
        </table>
    </a>
</div>
</p>
<h4>Suggestions</h4>
<p>

   <br>
    The problem indicates you requested large task memory (set <strong>mapreduce.map(or reduce).memory.mb</strong> > 2048), but the task average used physical memory is low.
   <br>
    You should try to decrease <strong>mapreduce.map(or reduce).memory.mb</strong>.
   <br>
    If you get OutOfMemory error, think about why such error happens before increase the memory back again.
    <br>
    For example, see if there is any mapper/reducer input data skew (some task processes larger input than others).
    <br>
    Also check if you are doing memory intensive operation at reducer side for e.g. count distinct. These operations can be rewritten using Secondary Sort concept (sort the values), which will reduce the memory consumption 
    <br>
    <strong>This is a new heuristic. We are still working on that. Any feedback is welcomed!</strong>
    <br>

    Examples on how to set the parameter:
    <ul>
    <li>HadoopJava: conf.setLong("mapreduce.map(or reduce).memory.mb",4096)</li>
    <li>Pig: set mapreduce.map(or reduce).memory.mb 4096 </li>
    <li>Hive: set mapreduce.map(or reduce).memory.mb=4096</li>
    </ul>

   <br>
    See <a href="https://github.com/linkedin/dr-elephant/wiki/Tuning-Tips">Hadoop Tuning Tips</a> for further information.<br>
